1. Field of the Invention
Typical embodiments of the invention are systems and methods for identifying black bars in images (e.g., images determined by frames of video data).
2. Background of the Invention
Throughout this disclosure including in the claims, the expression “black bar” is used to denote a block of contiguous pixels of an image determined by video data (e.g., one or more contiguous rows of pixels, or one or more contiguous columns of pixels of a video frame) which are at least substantially identical (e.g., which have the same, or substantially the same, color and luminance). For example, the pixels in a “black bar” may all be identical black pixels (having zero, or minimal luminance) or they may all have identical (or substantially identical) nonzero (or greater than minimal) luminance and identical (or substantially identical) color. A black bar may comprise non-identical pixels due to image noise, or other causes.
Throughout this disclosure including in the claims, the expression performing an operation “on” signals or data (e.g., filtering the signals or data) is used in a broad sense to denote performing the operation directly on the signals or data, or on processed versions of the signals or data (e.g., on versions of the signals that have undergone preliminary filtering prior to performance of the operation thereon).
Throughout this disclosure including in the claims, the expression “system” is used in a broad sense to denote a device, system, or subsystem. For example, a subsystem that implements a filter may be referred to as a filter system, and a system including such a subsystem (e.g., a system that generates X output signals in response to multiple inputs, in which the subsystem generates M of the inputs and the other X-M inputs are received from an external source) may also be referred to as a filter system.
Throughout this disclosure including in the claims, the noun “display” and the expression “display device” are used as synonyms to denote any device or system operable to display an image or to display video in response to an input signal. Examples of displays are computer monitors, television sets, portable devices such as tablets and phones and home entertainment system monitors or projectors.
Throughout this disclosure including in the claims, the term “processor” is used in a broad sense to denote a system or device programmable or otherwise configurable (e.g., with software or firmware) to perform operations on data (e.g., video or other image data). Examples of processors include a field-programmable gate array (or other configurable integrated circuit or chip set), a digital signal processor (e.g., a GPU) programmed and/or otherwise configured to perform pipelined processing on video or other image data, a programmable general purpose processor or computer, and a programmable microprocessor chip or chip set.
Legacy video material often contains black bars on the top and bottom (or left and right) of the displayed image to adjust the image size from one aspect ratio to another (e.g. from 16:9 to 4:3). However, when applying image processing algorithms to images containing black bars, it is typically not desirable to include the pixel values of the black bars into statistical or spatial computations. Additionally, some consumers detest black bars displayed with content and may wish a mode on their display device that automatically crops and scales images to fill the screen.
Several methods have been proposed for identifying black bars (sometimes referred to as constant blocks or black blocks) in video frames. For example, it has been proposed to compute a running average of the values of the pixels in a selected line (e.g., column) of an image, and to identify the location of a sharp change in the running average as the location of an edge (e.g., a row) of a black bar. It has also been proposed (e.g., in U.S. Pat. No. 7,538,821, issued May 26, 2009) to identify a black pixel that is contiguous to a black bar as belonging to the black bar, unless a dispersion value of a line of pixels (associated with the black pixel in question) exceeds a threshold value. If the dispersion value of the line of pixels exceeds the threshold value, the black pixel in question is assumed to belong to an image area (rather than to the black bar).
For another example, U.S. Pat. No. 6,061,400, issued May 9, 2000, describes a method in which the mean and standard deviation of the luminance values of each row of pixels of a frame are determined. If the standard deviation for a row is determined to be less than a threshold value (e.g., 5), the row is identified as a constant block. If two adjacent rows are identified as constant blocks, the difference between the mean values of the rows is calculated, and the rows are identified as belonging to the same constant block if the difference between the mean values is determined to be sufficiently small (e.g., less than 3% of the overall range of possible luminance values). U.S. Pat. No. 6,061,400 also proposes determining the strength of an identified edge of a constant block by summing the absolute differences between pairs of pixels on either side of the edge, and identifying the edge as a strong edge (rather than a weak edge) if the strength exceeds a predetermined threshold.
However, conventional methods for identifying black bars in video frames have not been robust against high levels of noise in the video data. The identification of black bars in well exposed and accurately processed digital images or video frames is relatively simple. However, when the image data (e.g., video data) indicative of these bars is degraded, conventional black bar identification methods do not reliably identify black bars in a manner that is both simple and accurate. For example, the conventional methods may incorrectly identify (as black bars) very low contrast rows and columns directly adjacent to black bars. Image data degradation can be caused by incorrect black levels, noise in the bars and blurred boundaries between image regions and the black bars.
In a class of embodiments, the invention is a method for detecting black bars in simple cases as well as under difficult conditions (e.g., in the presence of noise, blur and low contrast image margins). Typical embodiments are computationally efficient.